Goto

Collaborating Authors

 benchmarking machine learning


ClimateLearn: Benchmarking Machine Learning for Weather and Climate Modeling

Neural Information Processing Systems

Modeling weather and climate is an essential endeavor to understand the near-and long-term impacts of climate change, as well as to inform technology and policymaking for adaptation and mitigation efforts. In recent years, there has been a surging interest in applying data-driven methods based on machine learning for solving core problems such as weather forecasting and climate downscaling. Despite promising results, much of this progress has been impaired due to the lack of large-scale, open-source efforts for reproducibility, resulting in the use of inconsistent or underspecified datasets, training setups, and evaluations by both domain scientists and artificial intelligence researchers. We introduce ClimateLearn, an open-source PyTorch library that vastly simplifies the training and evaluation of machine learning models for data-driven climate science. ClimateLearn consists of holistic pipelines for dataset processing (e.g., ERA5, CMIP6, PRISM), implementing state-of-the-art deep learning models (e.g., Transformers, ResNets), and quantitative and qualitative evaluation for standard weather and climate modeling tasks. We supplement these functionalities with extensive documentation, contribution guides, and quickstart tutorials to expand access and promote community growth. We have also performed comprehensive forecasting and downscaling experiments to showcase the capabilities and key features of our library. To our knowledge, ClimateLearn is the first large-scale, open-source effort for bridging research in weather and climate modeling with modern machine learning systems.


Benchmarking machine learning for bowel sound pattern classification from tabular features to pretrained models

Mansour, Zahra, Uslar, Verena, Weyhe, Dirk, Hollosi, Danilo, Strodthoff, Nils

arXiv.org Artificial Intelligence

The development of electronic stethoscopes and wearable recording sensors opened the door to the automated analysis of bowel sound (BS) signals. This enables a data-driven analysis of bowel sound patterns, their interrelations, and their correlation to different pathologies. This work leverages a BS dataset collected from 16 healthy subjects that was annotated according to four established BS patterns. This dataset is used to evaluate the performance of machine learning models to detect and/or classify BS patterns. The selection of considered models covers models using tabular features, convolutional neural networks based on spectrograms and models pre-trained on large audio datasets. The results highlight the clear superiority of pre-trained models, particularly in detecting classes with few samples, achieving an AUC of 0.89 in distinguishing BS from non-BS using a HuBERT model and an AUC of 0.89 in differentiating bowel sound patterns using a Wav2Vec 2.0 model. These results pave the way for an improved understanding of bowel sounds in general and future machine-learning-driven diagnostic applications for gastrointestinal examinations


ClimateLearn: Benchmarking Machine Learning for Weather and Climate Modeling

Neural Information Processing Systems

Modeling weather and climate is an essential endeavor to understand the near- and long-term impacts of climate change, as well as to inform technology and policymaking for adaptation and mitigation efforts. In recent years, there has been a surging interest in applying data-driven methods based on machine learning for solving core problems such as weather forecasting and climate downscaling. Despite promising results, much of this progress has been impaired due to the lack of large-scale, open-source efforts for reproducibility, resulting in the use of inconsistent or underspecified datasets, training setups, and evaluations by both domain scientists and artificial intelligence researchers. We introduce ClimateLearn, an open-source PyTorch library that vastly simplifies the training and evaluation of machine learning models for data-driven climate science. ClimateLearn consists of holistic pipelines for dataset processing (e.g., ERA5, CMIP6, PRISM), implementing state-of-the-art deep learning models (e.g., Transformers, ResNets), and quantitative and qualitative evaluation for standard weather and climate modeling tasks.


PhilHumans: Benchmarking Machine Learning for Personal Health

Liventsev, Vadim, Kumar, Vivek, Susaiyah, Allmin Pradhap Singh, Wu, Zixiu, Rodin, Ivan, Yaar, Asfand, Balloccu, Simone, Beraziuk, Marharyta, Battiato, Sebastiano, Farinella, Giovanni Maria, Härmä, Aki, Helaoui, Rim, Petkovic, Milan, Recupero, Diego Reforgiato, Reiter, Ehud, Riboni, Daniele, Sterling, Raymond

arXiv.org Artificial Intelligence

Understaffing has been consistently identified as the major challenge facing Healthcare today [7, 1, 2, 21, 55, 82, 97, 87, 124]. Automation tools that make use of Machine Learning (also known as Healthcare 4.0 [126]) have been consistently identified as crucial for reducing the workload of Healthcare professionals and improving the quality of care [5, 34, 44, 46, 78, 86, 94, 136]. In turn, the shortage of standard benchmarks has been consistently identified as a central roadblock for machine learning in Healthcare [27, 31, 49, 52, 59, 76, 81, 95, 110]. Whether it's ImageNet [32] in Computer Vision or GLUE [128] in natural language processing, benchmarks are a core research tool in mature applications of machine learning, enabling quantitative analysis of learning methodologies to guide and orient their development.


Benchmarking Machine Learning on the New Raspberry Pi 4, Model B

#artificialintelligence

At the start of last month I sat down to benchmark the new generation of accelerator hardware intended to speed up machine learning inferencing on the edge. So I'd have a rough yardstick for comparison, I also ran the same benchmarks on the Raspberry Pi. Afterwards a lot of people complained that I should have been using TensorFlow Lite on the Raspberry Pi rather than full blown TensorFlow. They were right, it ran a lot faster. Then with the release of the AI2GO framework from Xnor.ai, which uses next generation binary weight models, I looked at the inferencing speeds of these next generation of models in comparison to'traditional' TensorFlow.